ai blog
Who invented deep residual learning?
Who invented deep residual learning? Modern AI is based on deep artificial neural networks (NNs). As of 2025, the most cited scientific article of the 21st century is an NN paper on deep residual learning with residual connections . Here is the timeline of the evolution of deep residual learning: 1991: recurrent residual connections (weight 1.0) solve the vanishing gradient problem 1997 LSTM: plain recurrent residual connections (weight 1.0) 1999 LSTM: gated recurrent residual connections (gates initially open: 1.0) 2005: unfolding LSTM--from recurrent to feedforward residual NNs May 2015: very deep Highway Net--gated feedforward residual connections (initially 1.0) Dec 2015: ResNet--like an open-gated Highway Net (or an unfolded 1997 LSTM) His recurrent residual connection was mathematically derived from first principles to overcome the fundamental deep learning problem of vanishing or exploding gradients, first identified and analyzed in the very same thesis. That is, at every time step of information processing, this unit just adds its current input to its previous activation value. The invariant residual connections transport error signals back to typically highly nonlinear adaptive parts of the NN where they can cause appropriate weight changes.
AI predicting the future • AI Blog
No one knows exactly what the future holds, but that isn't stopping artificial intelligence from trying to predict it. AI has been making some pretty impressive predictions lately, and if it continues on this trajectory, we're in for an exciting future. So what does AI foresee for the years to come? Read on to find out! If you had asked me this question a few years ago, I would have given you a very different answer than the one I would give today. Technology is evolving at an unprecedented rate, and it's hard to predict where it will go next.
What would Albert Einstein think of AI? • AI Blog
What would Albert Einstein think of AI? We may never know for sure, but it's fascinating to imagine. Some believe that he would have been a strong advocate for the technology, while others contend that he would have been more cautious about its implementation. No matter where you stand on this debate, one thing is for sure: AI is here to stay. And with its ever-growing presence in our lives, it's important to consider Einstein's potential thoughts on the matter.
Is AI ready to create all artwork for this blog? • AI Blog
Do you have a knack for design? Are your skills in creating the perfect poster, flyer, or social media graphic unmatched? You're going to be replaced by a computer. Artificial intelligence is getting better and better at creating graphics, and soon your job will be done by a machine. Don't worry though – you can still use your creative talents in other ways!
Is AI more Republican or Democrat? • AI Blog
The answer to this question may surprise you: AI is more Republican than Democrat. The study found that, when it comes to political values, AI is more closely aligned with the Republican Party than the Democratic Party. Specifically, the study found that AI is more likely to value individual liberty and free markets over government intervention and regulation. Additionally, AI is more likely to support traditional values such as patriotism and religion. Of course, it's important to remember that AI is not a person and cannot vote.
Is AI good for humanity? • AI Blog
The debate over whether artificial intelligence is good or bad for humanity is one that has been ongoing since the inception of AI itself. On one side, proponents argue that AI can help us to solve some of the world's most pressing problems, such as climate change, disease and poverty. On the other side, detractors claim that AI will eventually surpass human intelligence, leading to a future in which machines rule the world. So far, there is no clear answer as to which side is right. However, what is certain is that AI presents both risks and opportunities for humanity.
A conversation with Kevin Scott: What's next in AI - The AI Blog
Artificial intelligence systems powered by large language models today are transforming how people work and create, from generating lines of code for software developers to sketches for graphic designers. Kevin Scott, Microsoft's chief technology officer, expects these AI systems to continue to grow in sophistication and scale--from helping address global challenges such as climate change and childhood education to revolutionizing fields from healthcare and law to materials science and science fiction. Scott recently shared his thoughts with us on the impact of AI for knowledge workers and what's next in AI. In your mind, what were some of the most important advancements in AI this year? When we were heading into 2022, I think just about everybody in AI was anticipating really impressive things to take place over the next twelve or so months.
Action Selection • AI Blog
Action selection is the process of choosing which action to take in any given situation. This process is carried out by the brain, and it involves a complex interplay of various cognitive and emotional factors. In some cases, the decision may be made purely on the basis of logic; however, in other cases, emotions such as fear or excitement may also come into play. The specific algorithms used by the brain to choose an action are still not fully understood; however, it is clear that action selection is a vitally important part of human cognition. Action selection is what allows us to navigate the complex landscape of our lives and make choices that help us to achieve our goals.
Action Model Learning • AI Blog
Action model learning is a subfield of machine learning that focuses on learning how to perform actions in the world. This can include learning how to control robots, navigate in virtual environments, and much more. Action model learning focuses on giving machines the ability to learn from and predict actions. This can be done by observing and interactively exploring the environment, or by using reinforcement learning methods. Action model learning is an important area of research because it has the potential to create systems that can autonomously learn how to perform tasks by imitating human or animal behavior.